Could you elaborate a bit on how these reflect on the commits of the PR?

Static code tests uses dockerized services instead of mocked backends for DB, MQ and S3.
Functionality tests track the entire path from upload to complete ingestion and dataset release/deprecation.

also why is this needed?

Removes code and functionality tests for the sda-pipeline folder.

This could maybe benefit from being an epic or at least multiple PRs but it's a bit late now :)

I was also expecting a whole different thing from the title of the ticket. Are we archiving the sda-pipeline repo after this? Would that mean that we're not keeping any history of the merged repo?

Backup is merged with finalize in order to simplify the message flow.

so the finalize service also performs the backup if a backup storage is configured, do i understand this correctly ? seems to also include the sftp backup type which i assume is for mirroring

Precisely, the merge is primarily because there is no dedicated backup message.
sftp is included since it exists as a storage backed.

When deploying shall we set up one or more finalize services, each doing different functionalities ? i assume one (maybe 2 for mirroring use case ?), but want to be sure

This PR is targeted at the FEGA usecase and as such mirroring is not in the scope.
If a dedicated mirroring step is required in addition to backup we might have to bring the backup service back under a different name.

shall we include this in the description and maybe adjust finalize.md ?

Yes the readme needs to be updated.

Could you elaborate a bit on how these reflect on the commits of the PR?

Static code tests uses dockerized services instead of mocked backends for DB, MQ and S3.
Functionality tests track the entire path from upload to complete ingestion and dataset release/deprecation.

also why is this needed?

Static tests usually use mocked backeds, but mocking an MQ backend is all but impossible. Even the libray repo uses live backends in their tests.

Removes code and functionality tests for the sda-pipeline folder.

This could maybe benefit from being an epic or at least multiple PRs but it's a bit late now :)

I was also expecting a whole different thing from the title of the ticket. Are we archiving the sda-pipeline repo after this? Would that mean that we're not keeping any history of the merged repo?

How to deal with the history was solved when this repo was created, and all standalone sda-XXX repos that are merged into this will be archived when the merge of a repo is completed. Currently only sda-common and sda-s3proxy is fully merged.

Also unsure if I understand why sda-pipeline still remains with migrated contents here,

Also unsure if I understand why sda-pipeline still remains with migrated contents here,

Not every component is migrated yet and until that the old folder will still be around.

Will split into separate parts.